Manipulating an on-screen object using zones surrounding the object

ABSTRACT

A user interface for manipulating objects of various types in a consistent manner. Each on-screen object is surrounded by a control region including a number of zones for performing various control operations on the object. Multiple input modes are available for interacting with the zones, allowing object manipulation commands to be initiated in several different ways, such as via stroke input, pressing a mouse button, double-clicking, menu selection, voice input, and the like. The user interface is operable using any of several different types of input devices.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a divisional of U.S. Utility patentapplication Ser. No. 10/629,129 for “Manipulating an On-Screen ObjectUsing Zones Surrounding The Object,” filed on Jul. 28, 2003 which isrelated to U.S. Utility Patent No. 7,055,110 for “Common On-Screen ZoneFor Menu Activation And Stroke Input.” The disclosure of the relatedapplication is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to user interfaces for managingon-screen objects, and more particularly to a user interface thatprovides a consistent set of object management zones surrounding anon-screen object.

2. Description of the Background Art

Existing user interfaces provide many different techniques for moving,altering, controlling, and otherwise manipulating on-screen objects suchas windows, images, text blocks, video, and the like. For example, theWindows XP operating system, available from Microsoft Corporation ofRedmond, Wash., provides user interface mechanisms for manipulatingvarious types of on-screen objects. Examples of such user interfacemechanisms include:

-   -   application menus (e.g., click on an object to select it, and        select an operation from an application menu);    -   on-screen buttons (e.g., click on an object to select it, and        click a button to perform an operation on the object);    -   context-sensitive menus (e.g., right-click on an object and        select an operation from a pop-up menu);    -   resize borders or handles (e.g., click-drag a window edge or        object handle to resize the window or object); and    -   keyboard commands (e.g., click on an object to select it (or use        a keyboard to navigate to an on-screen object), and hit a        keystroke to perform an operation on the object).

One problem with most existing techniques is that there is no consistentuser interface paradigm for manipulating objects of different types. Forexample, the user interface for controlling text objects issignificantly different from the user interface for controlling graphicobjects. If a user wishes to resize a text object by increasing the textfont, he or she performs an entirely different action than he or shewould perform for resizing a graphic object. Accordingly, users mustlearn a variety of different manipulation methods, and know when toapply which method to which type of object. Often users becomedisoriented and confused when attempting to control certain types ofobjects, particularly when the user interface elements for the objectbeing controlled differ from those to which the user has becomeaccustomed.

Furthermore, existing techniques for activating certain objectmanipulation operations can be cumbersome, difficult to learn, orcounterintuitive. For example, specialized on-screen objects, such asobjects for representing time periods, often employ different userinterface paradigms that may be unfamiliar to users.

In addition, many such techniques do not translate well from one inputmechanism to another (such as pen-based, mouse, voice, and keyboardinput mechanisms). Users switching from one input mechanism to anothermust often learn a new object control paradigm in order to use thesoftware effectively.

What is needed, therefore, is a consistent, unified user interfaceparadigm for providing controls for manipulating on-screen objects,which addresses the limitations of conventional schemes. What is furtherneeded is a user interface paradigm that is extensible and thatfacilitates ease of use and ease of learning, even when the user isattempting to manipulate different types of objects. What is furtherneeded is a user interface paradigm that is usable with different typesof input mechanisms, and that facilitates transitions from one inputmechanism to another with minimal disruption, confusion, andre-learning.

SUMMARY OF THE INVENTION

In one aspect of the present invention, each object is surrounded by anumber of zones for performing various control operations on the object.The appearance of the zones can vary from one embodiment to another, butin general is kept consistent among various object types within aparticular user interface. The zones surrounding an object are referredto collectively as a control region. Each zone of the control region isassociated with a series of commands related to a particular type ofediting or manipulation function. Displaying the control region aroundthe object enables the user to immediately focus on and identify editingfunctions that can be performed on the object. Further, because of theproximity of the control region about the object, the user does not needto move the cursor to a distinct menu, tool palette, or other userinterface element, and as such, the user can select and execute thedesired command more quickly and efficiently.

In one embodiment, one control region is visible at a time,corresponding to the object that is currently active (that is, theobject that currently has focus); in another embodiment, control regionsfor some or all on-screen objects are visible at the same time.

In one embodiment, the control region for an object includes at leastnine zones that surround the object itself. The nine zones, includingone overlaying the object representation itself, form a three-by-threematrix. In one embodiment the object representation is located in thesame area as the center zone. Thus, eight zones surround the object,forming four corners and four sides, and one zone overlays the object.These zones provide additional control functionality for manipulatingthe object. In one embodiment, additional zones are provided, includingany or all of: a rectangular band-shaped zone located at the outsideedge of the center zone, another rectangular band-shaped zone located atthe outside edge of the control region, and a rectangular zone locatedalong an edge of the control region. One skilled in the art willrecognize that any number of zones may be provided, in any arrangementor layout, without departing from the essential characteristics of thepresent invention.

The user specifies commands for manipulating the on-screen object byinteracting with the zones within the object's control region.Interaction modes include, for example: pen (touch, press, stroke, orthe like); mouse; keyboard; voice; remote controller; and the like. Inone embodiment, the user interface of the present invention is capableof receiving user commands in at least two of the above modes, thusallowing a user to interact with the system in a consistent manner usingtwo or more input devices. Commands can be activated in different ways,including menu selection, strokes, clicks, button presses, and the like,performed in the area defined by the corresponding zone. In oneembodiment, available commands for interacting with an object includingboth geometric editing operations (i.e., those that involve moving,resizing, distorting, and/or rotating), and non-geometric editingoperations.

The invention thus provides an easy-to-use, understandable userinterface paradigm wherein the control region for each object surroundsthe object itself, provides access to all relevant editing operationswith respect to the object, and provides multiple input modes thatoperate consistently, and wherein interactions with zones are consistentfrom one object type to another.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an example of a control region for an on-screen object,according to one embodiment.

FIG. 1A depicts an example of a control region for an on-screen objectincluding additional rectangular band-shaped regions.

FIG. 2 depicts an example of a menu associated with a zone, according toone embodiment of the invention.

FIGS. 3A and 3B depict examples of command structures for the zones ofan object, according to one embodiment.

FIGS. 4A and 4B depict examples of a keyboard that includes additionalkeys corresponding to various zones.

FIG. 5 depicts an example of a remote controller for providing inputaccording to one embodiment of the invention.

FIG. 6 is a screen shot depicting an example of a menu for an inputfield.

FIGS. 7A and 7B depict an example of the operation of the presentinvention in connection with an input field.

FIG. 8 is a screen shot depicting examples of input objects.

FIG. 9 is a screen shot depicting an example of a menu for a table cell.

FIG. 10 is a screen shot depicting an example of a menu for a tablecolumn.

FIG. 11 is a screen shot depicting an example of a menu for a table row.

FIG. 12A depicts an example of a control region for a text object.

FIG. 12B depicts an example of a control region for a text object, thecontrol region including text labels.

FIG. 12C depicts an example of a control region for a text object, thecontrol region including icon labels.

FIG. 12D depicts an example of a control region for a text object, thecontrol region including a pop-up label.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

The following description of system components and operation is merelyexemplary of embodiments of the present invention. One skilled in theart will recognize that the various designs, implementations, andtechniques described herein may be used alone or in any combination, andthat many modifications and equivalent arrangements can be used.Accordingly, the following description is presented for purposes ofillustration, and is not intended to limit the invention to the preciseforms disclosed.

The invention may be implemented, for example, on a conventionalpersonal computer running any known operating system, such as MicrosoftWindows XP available from Microsoft Corporation of Redmond, Wash. Asdescribed below, input to the system of the invention can be provided byany of a number of types of devices, including for example: keyboard,mouse, touchscreen, touchpad, trackball, tablet, microphone, remotecontrol device, and/or the like. As will be seen from the followingdescription, the present invention provides techniques for acceptinguser inputs from various types of input devices and according to variousmodes in a consistent and intuitive manner.

One skilled in the art will recognize that the user interface techniquesof the present invention may be implemented on devices other thanpersonal computers in which a display is provided to the user. Forexample, such techniques may be implemented on cell phones, consumerelectronic devices, personal digital assistants, office appliances,wall-mounted controllers, automotive control systems, and the like. Thedescription of the invention herein as it might be implemented on apersonal computer is not intended to restrict the scope of the presentinvention or the range of devices on which it can operate.

User Interface Layout

Referring now to FIG. 1, there is shown an example of a user interface100 according to an embodiment of the present invention. User interface100 may be displayed, for example, on a display screen of a conventionalpersonal computer. Object 102 is an example of an on-screen objectdisplayed within user interface 100. In the example of FIG. 1, object102 is a graphic object. However, one skilled in the art will recognizethat object 102 can be a text object, or a representation of a video oraudio object, or any other type of object that can be displayed orrepresented within the context of user interface 100.

As shown in FIG. 1, according to one embodiment object 102 is surroundedby various zones 103(A-K), referred to collectively as a control region.Each zone 103 provides a designated type of functionality forcontrolling one or more aspects of object 102, or otherwise manipulatingobject 102. Object 102, together with the surrounding control regionincluding zones 103, form a window within user interface 100.

The area of user interface 100 surrounding the window is referred to asa “panel” 104. Any number of windows can be presented within panel 104simultaneously, and these windows can overlap one another, or they canbe tiled, minimized, or otherwise manipulated according to known userinterface techniques as modified and enhanced by the improvements of thepresent invention. In one embodiment, only one window is active at anygiven time, and this is the window for the object that has the focus orthat is currently active. The user can cause a window (and its object102) to be active according to well-known techniques such as: moving acursor to a window and clicking a button; or using a series ofkeystrokes or voice commands to select and activate a particular window.In one embodiment, zones 103 are displayed only for the currently activewindow, and are omitted for all other windows. In another embodiment,zones 103 are displayed for all on-screen windows. In addition, in oneembodiment panel 104 itself is treated like any other window, so that itcan be activated and zones 103 relevant to the control and manipulationof panel 104 can be displayed. Also shown is cursor 110, which iscontrolled by the user via a mouse or other pointing device, and can beused to select and activate commands and menus as described below.

In the example of FIG. 1, ten zones 103A through 103K are shown,including zone 103J that overlays object 102, and zone 103K that islocated outside control region 101. Zone 103K is depicted in thedrawings as extending along the bottom of region 101, but in oneembodiment includes all areas not within region 101. One skilled in theart will recognize that any number of zones could be included, and thatthe particular arrangement of zones 103 shown in FIG. 1 is merelyexemplary. For example, as shown in FIG. 1A, additional rectangularband-shaped regions 103L, 103M could be located at the outer edge ofzone 103J and at the outer edge of control region 101.

Input Mechanisms

In operation, the device on which the user interface 100 operatesdetects user commands with respect to the various zones 103 of anonscreen object 102. For example, a user may use a mouse or other inputdevice to move an onscreen cursor 110 to a zone 103. The user may thenperform an action, such as clicking a button on a mouse, to activate acommand or operation associated with the zone 103. As will be describedin more detail below, some zones 103 may have different commands oroperations associated with different user actions; for example, a usermay click a left button to activate a first command, or a right buttonto activate a second command. Other possible actions associated withdifferent types of commands include: “stroking” within a zone 103 byholding down a button and moving the cursor 110 a short distance in aparticular direction; “shift-clicking” within a zone 103 by holding downa shift key (or other modifier key) while clicking within the zone 103;or double- or triple-clicking within a zone 103.

Any of the above-described input devices can be used for interactingwith zones 103. Examples of user action and input mechanisms forinteracting with zones 103 include:

-   -   Pen input: Suitable for handwriting input, stroking, gestures,        symbols, and abbreviations;    -   Mouse or other pointing device: Suitable for clicking, dragging,        stroking, and the like;    -   Keyboard: Suitable for text entry, navigation among onscreen        elements, or single-key activation of commands;    -   Voice input: Suitable for activating commands and for inputting        text. The user speaks the name of a zone (e.g., “copy”) to        display a menu for that zone, and then speaks a command from the        displayed menu to activate that command.    -   Remote control (e.g., RF or IR transmitter) input: Suitable for        navigation, data entry, command activation, and possibly cursive        writing and stroking.

In one embodiment, the user interface of the present invention iscapable of receiving user commands in a consistent manner in two or moreof the above modes.

User Actions

Many different command sets are possible. In one embodiment, the usercan perform the following user input actions to activate commands:

-   -   Click standard button (e.g. left-click) while cursor 110 is in        zone 103 (or, for pen input, touch zone 103 without pressing pen        button): activates a pop-up menu, which may be also called a        primary menu, containing commands for zone 103. The displayed        pop-up menu shows the relevant commands for the zone 103, and        allows the user to navigate and select one of the displayed        commands using known menu navigation techniques. In one        embodiment, each pop-up menu for each zone 103 is associated        with a particular theme or general type of action, or group of        associated actions. For example, a zone 103 may have a variety        of commands or actions related to font characteristics; another        zone 103 may have a variety of commands related to paragraph        characteristics, and so forth. In one embodiment, included in        the pop-up menu are commands that can be invoked by movement of        the input device by a stroke in a predetermined direction (e.g.,        up, down, left, or right). FIG. 2 illustrates a pop-up menu        showing four stroke icons 802, each stroke direction        corresponding to a particular command. Also included is a        command that corresponds to a double-click for zone 103. One        skilled in the art will recognize that the menu can include        commands for any other type of action or gesture that can be        performed within zone 103, and may also include commands that        cannot be activated directly using an action or a gesture.        Referring now to FIG. 2, there is shown an example of such a        menu 801, including commands 803 and icons 802 indicating        corresponding stroke directions, and commands 803 having icons        802 indicating other actions such as a double-click and an Enter        key.    -   Click alternate button (e.g. right-click), or modified button        (e.g. shift-click), or click and hold button while cursor 110 is        in zone 103 (or, for pen input, either touch zone 103 with pen        button pressed, or touch and hold the pen in zone 103):        activates an alternate pop-up menu, which may also be called a        secondary menu. The alternate pop-up menu operates in a similar        manner as the first pop-up menu 801, and also includes icons or        indicators showing direct actions that correspond to stroke        directions, double-click, and the like, where applicable. In one        embodiment, the on-screen cursor 110 changes to indicate that        the button has been held long enough to activate the alternate        menu.    -   Stroke (i.e., hold down button and move cursor 110 a short        distance in a particular direction) starting in zone 103 (or,        for pen input, stroke without pressing pen button): performs an        action associated with the particular stroke direction. Strokes        can be of any type, including straight lines, curves, or more        complex shapes or gestures. In one embodiment, as described        herein, strokes are associated with particular directions (up,        down, left, or right), although one skilled in the art will        recognize that other types of strokes may also be used. As        described above, in one embodiment, the pop-up menu 801 for a        zone 103 includes each of the stroke directions and indicates        what command is associated with each stroke direction. Thus, the        user can activate any of these actions by performing the stroke        input or by clicking and then selecting the desired item from        the pop-up menu 801. Longer strokes can be associated with a        click-and-drag action.    -   Double-click (or other action) in zone 103: performs an        associated command. As described above, in one embodiment, the        pop-up menu 801 for a zone 103 indicates what command is        associated with double-clicking (or other actions). Thus, the        user can activate the command by double-clicking or by clicking        and then selecting the desired item from the pop-up menu 801.    -   Alternate stroke (i.e., hold down alternate button (right        button), or modified button (shift-stroke), and move cursor 110        a short distance in a particular direction) starting in zone 103        (or, for pen input, stroke while pressing pen button, or hold        pen in place for a moment and then stroke): performs an        alternate command associated with the particular stroke        direction (up, down, left, or right). As described above, in one        embodiment, the alternate pop-up menu for a zone 103 includes        each of the stroke directions and indicates what alternate        action is associated with each stroke direction. In one        embodiment, the on-screen cursor 110 changes when the modifier        key is pressed, to remind the user that the alternate command        will be performed.    -   Alternate double-click (i.e., hold down alternate button (right        button), or modified button (shift-stroke), and double-click) in        zone 103: performs an associated command. As described above, in        one embodiment, the pop-up menu 801 for a zone 103 indicates        what command is associated with alternate double-clicking (or        other actions). Thus, the user can activate the command by        alternate double-clicking or by clicking and then selecting the        desired item from the pop-up menu 801.

Alternate menus, stroke commands, double-clicks, and the like can beactivated in a number of different ways, depending on the input device.For two-button devices, the alternate button can be used to activate thealternate commands. For one-button devices, the alternate menus andstrokes can be activated by pressing and holding the button down for amoment without movement. In one embodiment, the on-screen cursor 110changes to indicate when the alternate action will be performed. Amechanism for keyboard activation of commands is described in moredetail below.

Examples of Command Structures

Referring now to FIGS. 3A and 3B, there are shown examples of commandstructures according to one embodiment, based on the zone layout shownin FIG. 1. FIG. 3A depicts command types corresponding to primary menus,and FIG. 3B depicts command types corresponding to secondary, oralternate, menus.

In general, each zone 103 is associated with a number of actions thatare related to a particular function or type, as set forth below.Although these actions are in general consistent among object types,some actions may be object-specific and/or inapplicable to certain typesof objects.

In addition to the most commonly used actions that are generallyaccessible via a primary menu 801, each zone 103 is also associated withan alternate action or set of actions. For example, an alternate menumay be accessed by right-clicking or option-clicking a mouse, or bypressing a shift key prior to pressing a zone 103 activation key.Alternate keystrokes can be accessed by stroking while holding down aright mouse button, or a modifier key such as shift or option. Othertechniques may also be used for accessing alternate actions.

The following tables summarize primary command menus for the variouszones 103 according to one embodiment. For each type of object listed, anumber of available commands are shown; these commands may be availablevia a primary menu 801 and/or via stroke commands, double-clicks orother actions, as indicated on the table with symbols such as << (leftstroke), >> (right stroke), ^(up stroke), v (down stroke), and :(double-click).

Upper Left Zone 103A: Start Object Commands Document Start (changestop-left corner of document): << Left (unit panel) >> Right (unit panel)/\ Up (unit panel) \/ Down (unit panel) : Reset While browsing orentering characters: Start (changes first character): Text << Remove(leading character) File >> Add (leading blank) Date /\ Single quoteImage (entering characters only) \/ Cedilla (,) Music : Reset (removespecial characters) Video Parenthesis Message Bracket Curly bracketWhile playing: Start (changes playing position): Music << Left (second)Video >> Right (second) Message /\ Up (frame 1/30 s) \/ Down (frame 1/30s) : Reset (start to 0:00:00) Image Start (changes top-left cornercropping): << Left (pixel) >> Right (pixel) /\ Up (pixel) \/ Down(pixel) : Reset Shape Start (changes top-left extent of Shape): << Left(pixel) >> Right (pixel) /\ Up (pixel) \/ Down (pixel) : Reset (align togrid) Overlay objects (excludes: Document, Column, Start (changestop-left extent of Object): Row, Cell) << Left (pixel) >> Right (pixel)/\ Up (pixel) \/ Down (pixel) : Reset (align to grid) Paragraph Start(changes top-left margin): List << Left (character) Item in List >>Right (character) Table /\ Up (0.2 line) \/ Down (0.2 line) : Reset DateStart (Date object): << Left (back 1 month) >> Right (forward 1 month)/\ Up (forward 1 day) \/ Down (back 1 day) : Today Yesterday TomorrowNumber Start (numeric value): << Left (much smaller) >> Right (muchlarger) /\ Up (larger) \/ Down (smaller) : Round (zero last non-zerodigit) Time Start (start time): << Left (qtr.hour) >> Right (qtr.hour))/\ Up (minute) \/ Down (minute) Column Not Available Row Cell

Upper Middle Zone 103B: Insert Object Commands While browsing: Insert:Document << Insert contents of clipboard before current Text objectFile >> Insert contents of clipboard after current Date object Image \/Add current object to clipboard Music /\ Replace clipboard with copy ofcurrent object Video : Save (link names for later insert) Message Linkname 1 Shape Link name 2 Number . . . Time Link name n Paragraph ListItem in List Table Row Column Cell While entering Insert: characters: <<Insert contents of clipboard before cursor Text position File >> Insertcontents of clipboard after cursor Image position Music /\ Replaceclipboard with text of current object Video \/ Add text of currentobject to clipboard Message Insert (Clipboard with 1 objects): << Insertcontents of clipboard before current selection >> Insert contents ofclipboard after current selection /\ Replace clipboard with currentselection \/ Add current selection to clipboard

Upper Right Zone 103C: Delete Object Commands Document: Delete (entiredocument): /\ Replace (Clipboard with document) \/ Add (to Clipboard) :This While browsing: Delete (objects): Text << Before (this object)File >> After (this object) Date /\ Replace (Clipboard with this object)Image \/ Add (this object to Clipboard) Music : This (object) VideoMessage Shape Number Time While entering characters: Delete(characters): Text << Before (cursor) File >> After (cursor) Image /\Replace (Clipboard with text of object) Music \/ Add (text of object toClipboard) Video : This (adjacent word) Message Delete (characters): <<Before (selection) >> After (selection) /\ Replace (Clipboard withselection) \/ Add (selection to Clipboard) : This (selection) Paragraph:Delete (objects): List << Before (this object) Item in List >> After(this object) Table /\ Replace (Clipboard with this object) Row \/ Add(this object to Clipboard) Column : This (object) Cell Content (keepthis object)

Middle Left Zone 103D: Format Object Commands Document: Format: Text <<Cross out (Strike-through) File >> Italic Date /\ Bold Image \\Underline Music : Reset (normal) Video Save (format) Message Apply(saved format) Number Apply current format to preceding object TimeApply current format to successive object Paragraph List Item in ListTable Row Column Cell Shape: Format: Line << Starting arrow Arc >>Ending arrow Scribble Save (format) Apply (saved format) Apply currentformat to preceding object Apply current format to successive objectShape: Not available Rectangle Oval Point

Middle Right Zone 103E: Create Object Commands While browsing: Create(new object): Document << Before (same) Text >> After (same) File /\Parent (same as parent) Date \/ Child (same as last child) Image : TextMusic Time Video Number Message Scribble (Shape) Shape Line (Shape)Number Rectangle (Shape) Time Arc (Shape) Paragraph Oval (Shape) ListPoint (Shape) Item in List Paragraph Table Group Row List Column TableCell Date File Image Message Music Video While entering Create(abbreviation): characters: Text << Collapsed (long -> short) File >>Expanded (short -> long) Image /\ Deletion (from abbreviation list)Music \/ Addition (short|long) Video : All (expand all abbreviations)Message Long 1 (short 1) Long 2 (short 2) . . . Long n (short n) Overlayobjects (excludes: Create (new object): Document, Column, Row, << Left(same to left) Cell) >> Right (same to right) /\ Up (same above) \/ Down(same below) : Text Time Number Scribble (Shape) Line (Shape) Rectangle(Shape) Arc (Shape) Oval (Shape) Point (Shape) Paragraph Group ListTable Date File Image Message Music Video

Lower Left Zone 103F: Change To Object Commands Document Change(Bookmark) to: << Remove >> Add /\ Previous (first) \/ Next (last) :Reset (load Resource file) Bookmark 1 Bookmark 2 . . . Bookmark n Whilebrowsing or entering Change (characters) to: characters: Text << Firstletter CAP File >> Title case Image (entering characters only) /\Uppercase Music \/ Lowercase Video : Reset (automatic CAPS) MessageWhile playing: Change (playing position) to: Music << Fast reverse (15seconds) Video >> Fast forward (15 second) Message /\ Previous (clip) \/Next (clip) : Reset (to start of clip) Random (order of play) Sequential(order of play) Date Change (Date) to: << Lower (year) >> Higher (year)/\ Previous (type) \/ (type) : Reset (Aug. 10, 2003) Month day (August10) Long (Aug. 10, 2003) Day of week (Sunday Aug. 10, 2003) Concise(Aug. 10, 2003) Minimum (8/10) Paragraph Change (Paragraph) to: List <<Left (objects flow to right) Item in List >> Right (objects flow toleft) Table : Reset (no flow) Title case (override all) Uppercase(override all) Lowercase (override all) None (remove CAPS override) RowChange to: Column Title case (override all) Cell Uppercase (overrideall) Lowercase (override all) None (remove CAPS override) Number Change(Number) to: << Smaller (divide by 10) >> Larger (multiply by 10) /\Previous (type) \/ Next (type) : Reset Normal (Integer or decimal)Scientific notation Percent Dollars Time Change (alarm start time) to:<< Left (qtr.hour) >> Right (qtr.hour)) /\ Up (minute) \/ (minute) :Reset (remove alarm) Shape Change (rotation) Image

Lower Middle Zone 103G: Shift Object Commands While browsing: Shift:Document << First (child object) Paragraph >> Last (child object) List/\ Parent Item in List \/ Child Table : Play mode Column Row Cell Whileentering characters: Shift: Text << First (char. text entry) File >>Last (char. text entry) Image /\ Parent Music : Play mode Video MessageWhile playing: Shift (playing position): Music << Left (second) Video >>Right (second) Message /\ Up (frame 1/30 s) \/ Down (frame 1/30 s) :Reset (to start of clip) Shape Shift (opacity = 36%): << Less(opaque) >> More (opaque) /\ Transparent \/ Opaque Number Not AvailableTime

Lower Right Zone 103H: End Object Commands Document End (changesbottom-right corner of document): << Left (unit panel) >> Right (unitpanel) /\ Up (unit panel) \\ Down (unit panel) : Reset While browsing orentering End (changes terminating character): characters: Text << RemoveFile >> Question mark Date /\ Exclamation point Image (entering textonly) \\ Semicolon Music : Period Video Single quote (surroundingobject) Message Double quote (surrounding object) Parenthesis(surrounding object) Bracket (surrounding object) Curly bracket(surrounding object) Copyright (©) Registered (trademark ®) Whileplaying: End (changes playing position): Music << Left (second) Video >>Right (second) Message /\ Up (frame 1/30 s) \\ Down (frame 1/30 s) :Reset (to end of clip) Image End (changes bottom-right corner cropping):<< Left (pixel) >> Right (pixel) /\ Up (pixel) \\ Down (pixel) : Reset(remove cropping) Shape End (changes bottom-right extent of Shape): <<Left (pixel) >> Right (pixel) /\ Up (pixel) \\ Down (pixel) : Reset(align to grid) Overlay objects (excludes: End (changes bottom-rightextent of Object): Document, Column, << Left (pixel) Row, Cell) >> Right(pixel) /\ Up (pixel) \\ Down (pixel) : Reset (align to grid) ParagraphEnd (changes bottom-right margins): List << Left (character) Item inList >> Right (character) Table /\ Up (0.2 line) \\ Down (0.2 line) :Reset Column End (changes column width, absolute): << Left (0.1character) >> Right (0.1 character) : Reset (to automatic width) Convert(to % width) Before (apply same width) After (apply same width) End(changes column width, relative): << Left (0.1%) >> Right (0.1%) : Reset(to automatic width) Convert (to character width) Before (apply samewidth) After (apply same width) Row End (changes Row height, absolute):/\ Up (0.1 line) \\ Down (0.1 line) : Reset (to automatic height)Convert (to % height) Before (apply same height) After (apply sameheight) End (changes Row height, relative): /\ Up (0.1%) \\ Down (0.1%): Reset (to automatic height) Convert (to character height) Before(apply same height) After (apply same height) Cell End (changes numberof rows/columns spanned by cell): << Left (reduce cell span by onecolumn) >> Right (extend cell span by one column) /\ Up (reduce cellspan by one row) \\ Down (extend cell span by one row) : Reset (to 1column and 1 row) Number End (changes width): << Left (1 characternarrower) >> Right (1 character wider) /\ Reset (to standard width) TimeEnd (changes duration): << Left (qtr. hour less) >> Right (qtr. hourmore) /\ Up (1 minute more) \\ Down (1 minute less)

Center Zone 103J: Drag Object Commands Document Drag (document): << Left(unit panel) >> Right (unit panel) /\ Up (unit panel) \/ Down (unitpanel) : Enter (activate child object) While entering characters: Selectcharacters or insertion point Text File Date Image Music Video MessageText (while browsing) Drag: << Before >> After : Enter (start textentry) While browsing: Drag: File << Before Date >> After Image : Enter(load object) Music Video Message While playing: Drag (Play mode): Image: Exit Music Video Message Absolute positioned objects (excl. Drag(Overlay object): Document, Item in << Left (pixels) List, Column, Row,Cell) >> Right (pixels) /\ Up (pixels) \/ Down (pixels) : Enter(activate child object) Paragraph Drag: Item in List << Before Cell >>After : Enter (activate child object) Drag (empty object): : Enter(create Text object) Table Drag: Row << Before Column >> After List :Enter (activate child object) Number Drag: Time << Before Shape(In-flow) >> AfterOutside Zone 103K: Go To

If the user clicks on a control region 101 outside the currently activecontrol region 101, the object corresponding to the selected controlregion 101 is made active.

In one embodiment, whenever the user clicks at a location not within acontrol region 101 of an existing object, a new copy of a “resource”document is loaded into the empty space. The resource document is thehighest-level object in the object hierarchy.

If the user drags the cursor 110 across a series of objects, theseobjects are highlighted. When the button is released or the pen islifted, the most recently high-lighted object is activated and a newcontrol region 101 appears surrounding this object. If the user movesthe cursor 10 beyond the limits of the active object, the contents ofthe object are automatically scrolled to make additional objectsaccessible for activation.

Additional navigation among objects is available by activating a menucorresponding to outside zone 103K, or by performing a stroke or otheraction as listed below, while the cursor 110 is located in outside zone103K.

-   -   Go to:    -   First (sibling)    -   Last (sibling)    -   << Previous    -   >> Next    -   Parent    -   Child    -   : This (Document)    -   Resource 1    -   Resource 2    -   . . .    -   Resource n

Resources 1 through n are defined in the “Resource” document and specifyall major groupings of user information, hardware devices, and networkresources, as described in more detail below. Typical resourcedefinitions include:

-   -   Preferences    -   House    -   Media    -   Address    -   Name    -   EMail    -   Date    -   Subject    -   Phone    -   Web    -   Offline Storage    -   Printers

The following tables summarize alternate command menus for the variouszones 103 according to one embodiment. For each type of object listed, anumber of available commands are shown; these commands may be availablevia an alternate menu (activated, for example, by right-clicking) and/orvia alternate stroke commands, alternate double-clicks or other actions,as indicated on the table with symbols such as << (left stroke), >>(right stroke), ^ (up stroke), v (down stroke), and : (double-click).

Upper Left Zone 103A (Alternate): Fill Color

Fill color can be applied to any objects except for those that have nointerior space (such as lines, arcs, and points). Activating the fillcolor menu causes a palette to appear, from which the user can selectthe desired color.

Upper Middle Zone 103B (Alternate): Zoom Object Commands Document Zoom:Text << Smaller File >> Larger Date /\ Previous (font) Music \/ Next(font) Message : Reset (standard size and font) Number Serif TimeCursive Paragraph Monospace List Item in List Table Row Column CellImage Zoom: Video << Smaller >> Larger /\ Thumbnail : Reset (revert tooriginal size) Shape: Zoom: Rectangle << Smaller Oval >> Larger Line ArcPoint Shape: Zoom: Scribble << Smaller >> Larger : Reset (revert tooriginal size)Upper Right Zone 103C (Alternate): Undo

Alternate-clicking within zone 103C activates an undo command. The usercan also activate the alternate menu to navigate within the history ofcommands, as follows:

-   -   Undo (2 History objects):    -   << Older (show History)    -   >> Newer (show History)    -   Show all    -   All    -   : This (most recent History)        Middle Left Zone 103D (Alternate): Text Color

Text color can be applied to any object that includes text. Activatingthe text color menu causes a palette to appear, from which the user canselect the desired color.

Middle Right Zone 103E (Alternate): Transform To Object CommandsDocument Transform to: Protected Unprotected Cleaned up Original versionSave Text Transform to: << Combine >> Split File Date File Transformto: >> Split

Displayed content Text Date Transform to: >> Split

Displayed content Text Image Transform to: Music >> Split Video

Displayed content Message Shape-Line Transform to: Rectangle OvalShape-Arc Transform to: Line Rectangle Oval Shape-Scribble Transform to:Line Rectangle Oval Shape-Rectangle Transform to: Line Oval Shape-OvalTransform to: Line Rectangle Shape-Point Transform to: Oval ParagraphTransform to: List << Combine Item in List >> Split Table

Displayed content List Column Transform to: Row << Combine >> Split CellTransform to: << Combine >> Split List Number Transform to: Time >>Split TextLower Left Zone 103F (Alternate): Border Color

Border color can be applied to any object potentially having a border.This includes, for example, images, numbers, times, paragraphs, lists,items in lists, tables, and cells. Activating the border color menucauses a palette to appear, from which the user can select the desiredcolor.

Lower Middle Zone 103G (Alternate): Align Object Commands Document Notavailable Music Video Message Image Text Align (text vertically): File

Up (superscript) Date

Down (subscript) : Reset (normal) Overlay (NOT in-flow positioning)Before (same as this) After (same as this) Shape Align (to grid): <<Left >> Right

Up (larger grid size)

Down (smaller grid size) : Reset (normal) In-flow (Not overlaypositioning) Before (same as this) After (same as this) Overlay objects(excludes: Align (to grid): Document, Column, << Left Row, Cell) >>Right

Up (larger grid size)

Down (smaller grid size) : Reset (normal) In-flow (Not overlaypositioning) Before (same as this) After (same as this) Paragraph Align(contained objects horizontally): List << Left Item in List >> RightTable

Justify text (and left align)

Center : Reset (normal) Overlay (NOT in-flow positioning) Before (sameas this) After (same as this) Row Align (cell content horizontal &vertical): Column << Left Cell >> Right

Top

Bottom : Reset (normal) Center (horizontal & vertical) Horizontal centerVertical center Before (same as this) After (same as this) Number Align(decimal point position): << Left >> Right : Reset (normal) Overlay (NOTin-flow positioning) Before (same as this) After (same as this) TimeAlign (object): Overlay (NOT in-flow positioning) Before (same as this)After (same as this)

Lower Right Zone 103H (Alternate): Border Object Commands Document Notavailable Text File Date Music Video Message Image Border (surroundingobject): Number << Thinner Time >> Thicker Paragraph

Previous (border type) List

Next (border type) Item in List : Reset (standard border) Table NoneCell Ridge Solid Double Groove Inset (current) Outset Dashed Dotted RowBorder (between rows):

Add : Reset (between columns & rows) Border (between rows):

Remove : Reset (between columns & rows) Column Border (between columns):

Add : Reset (between columns & rows) Border (between columns):

Remove : Reset (between columns & rows) Shape Border (line type & width= 4): << Thinner >> Thicker

Previous (border type)

Next (border type) : Reset (single, solid, width = 2) Single (current)Double Triple Solid (current) Dotted Dashed Dot Dash

Center Zone 103J (Alternate): Copy

When the user alternate-clicks within zone 103J, a copy of the activeobject is copied by moving the outline of the active object to anotherdestination. The destination object can be in another document.

The user can activate a menu for positioning and manipulating the copyof the currently active object. The menu includes the following commands(some or all of which are also available via strokes):

-   -   Copy (move outline to destination):    -   Up (pixels)    -   Down (pixels)    -   << Left (pixels)    -   >> Right (pixels)    -   : Exit (activate this document)        Outside Zone 103K (Alternate): Scroll

Scroll commands are accessed by alternate-clicking outside controlregion 101. As long as the button is held or the pen stays in contactwith the display, the currently active object is directly scrolled usingthe pen or mouse movement. In one embodiment, as the scrolling limits ofthe object are reached, the object's outer edge (corresponding to thedirection of scrolling) changes color.

In another embodiment, an alternate-click activates a scroll menucontaining the following commands:

-   -   Scroll (document):    -   << Left (half page)    -   >> Right (half page)    -   Up (half page)    -   Down (half page)    -   : Reset (top or bottom)

As with the other menus described above, commands can also be directlyactivated by performing the stroke, double-click, or other actionindicated.

Wherever possible, each zone provides unified, consistent access to agroup of commands that are related to a particular theme. The inputactions of clicking, click-and-hold, or stroking provide various ways toaccess commands within the group.

For example, zone 103A operates as follows in connection with objects ofa time-varying nature (such as audio, video, animations, and the like).Zone 103A provides commands for changing the start of the data object(for example, if the data object is a video stream, zone 103A allowsmanipulation of the time code at which the video stream begins). Asdescribed in the table above, stroking in zone 103A changes the startpoint by an amount and direction that depend on the direction of thestroke. For example, vertical strokes would cause coarse adjustments tothe start point to be made (changing the minutes), while horizontalstrokes would cause fine adjustments to be made (changing the seconds).Alternatively, vertical strokes might change the value by a one-hourincrement (or some other relatively coarse increment), while horizontalstrokes might change the value by a one-minute increment (or some otherrelatively fine increment). In general, strokes in the up and leftdirections move the start point earlier, while strokes in the down andright directions move the start point later. The particular incrementsand directions of movement can be preset, or can be user-configurablevia a preferences or options screen.

Example Input Field

Referring now to FIGS. 7A and 7B, there is shown an example of theoperation of the present invention in connection with a particular typeof object: an input field 700. FIG. 7A depicts input field 700 beforecontrol region 101 has been made active (for example, before input field700 has focus or has been selected). The example input field 700 of FIG.7A indicates a time period (9:00-9:45), as may be included for examplein a calendaring program for maintaining information about appointmentsand reminders. Input field 700 is an object that has two primarypurposes: 1) to display a representation of a time period; and 2) toaccept user input for changing the time period and/or the manner inwhich it is displayed.

In one embodiment, input field 700 is made active (or has “focus”) whena user clicks or taps within its bounds or otherwise selects the field.As shown in FIG. 7B, control region 101, including zones 103, thenbecomes visible. The example of FIG. 7B illustrates how zones 103,described in general terms above, provide specific functionality forfacilitating user input to change the value and format for the timeperiod represented by input field 700. Specifically:

-   -   Zone 103A allows the user to change the start of the time        period. As described above, vertical strokes indicate coarse        changes, while horizontal strokes indicate fine changes. In one        embodiment, the time period adjustment is repeated if the user        holds the stylus or other pointing device in position at the end        of the stroke; in another embodiment, the time period adjustment        is repeated if the user continues to stroke in the same        direction (if desired, a wait period can be implemented before        the repetition commences, as is conventionally done in        auto-repeat modes for keyboards). The time period adjustment        repetition can be set to accelerate if the user continues to        hold for a predetermined period of time. In one embodiment, the        user can “back up” if the time period adjustment goes to far.        The display of input field 700 is continually adjusted so that        the user can see the time period adjustments in real-time and        release or backtrack at precisely the correct moment.    -   Zone 103H allows the user to change the end point of the time        period. End point adjustment is performed using the same        coarse/fine adjustment paradigm described above in connection        with zone 103A.    -   Zones 103B, 103C, 103D, 103E, 103F, 103G, and 103J provide        commands as described above.

In one embodiment, the duration of the event remains constant when thestart point is changed (so that the end point changes by the same amountas the start point). In another embodiment, the end point is heldconstant when the start point is changes (so that the duration changes).In yet another embodiment, the user can shift-click or indicate by amodifier key which of these two modes of operation is desired;alternatively, the mode can be set via a preferences or options screen.

Example Tables and Cells

The present invention provides a consistent interface for manipulatingthe characteristics of table objects and cell objects. Sizes, such asrow heights, column widths, and overall table sizes, can be specified inabsolute terms or as a percentage of a containing object. For example, acell width can be configured to be 25% of the width of the containingtable; if the table width is increased, the cell width increasesproportionately. The user interface allows for selection and operationon individual cells, rows, columns, or the entire table. In oneembodiment, for example, clicking on a corner of the table selects theentire table; clicking on the left or right edge selects the table rowat the clicked location; clicking on the top or bottom edge selects thetable column at the clicked location; and clicking on a cell selects theindividual cell. In one embodiment, the control region is presented forthe selected cell, column, row, or table, so that the user can thenactivate commands within the various zones of the active control region.

Keyboard Input

In addition to the input modes described above, the user can alsoactivate commands via a standard keyboard such as a QWERTY keyboard. Inone embodiment, each zone 103 is associated with a keystroke; strikingthe corresponding key causes the menu 801 for that zone 103 to bedisplayed. Stroke commands, double-click commands, and the like can alsobe associated with keystrokes, so that the user can activate the commandby striking the corresponding key (accompanied, perhaps, by a Controlkey, an Alt key, or some other modifier key). Letters, or otherindicators, can be displayed within zones 103 to show which keystrokecorresponds to each zone 103. Key-strokes for zones 103 can beconfigured in a mnemonically suitable manner (for example, Ctrl-Gcorresponds to Go To, while Ctrl-C corresponds to copy).

Referring now to FIGS. 4A and 4B, there is shown an example of akeyboard 400 that includes additional keys 401 corresponding to variouszones 103. Keys 401 are positioned around the perimeter of a standardQWERTY keyboard 403, with each key 401 in a location that maps to thecorresponding zone 103. For example, key 401A, which activates the startmenu 801, is located at the top left corner of keyboard 400, a locationthat corresponds to the location of zone 103A that also activates thestart menu 801. Similar correspondences apply for keys 401B through401H. Alternate menus can be activated, for example, by holding downshift key 404 while pressing the desired zone key 401.

In the example of FIG. 4A, keyboard 400 also includes keys 402U, 402D,402L, 402R that, when pressed, perform stroke actions corresponding toup, down, left, and right strokes. A user can press a key 401 followedby a stroke key 402 to per-form an action in the zone 103 correspondingto the pressed key 401. In one embodiment, when the user presses key401, the system waits a short, predetermined period of time (such as ¼second) before presenting the menu associated with key 401. If, duringthat waiting period, the user presses one of keys 402U, 402D, 402L,402R, or if he or she presses key 401 again, the command associated withthe specified stroke or double-click is immediately performed and themenu is not displayed. If, during the waiting period, the user pressesan alphabetic key indicating a desired command from the menu, thecommand is executed immediately, again without the menu itself beingdisplayed. If the user performs neither of these actions during thewaiting period, the system proceeds with display of the menu. In thismanner, the system accommodates expert users (who are familiar with thecommand structure and do not need to see the menus before activatingcommands within them), and novices (who wish to see the availablecommands before activating one).

Alternatively, an alternate stroke action can be activated by pressingshift plus a key 401 followed by a stroke key 402. Stroke keys 402 canbe configured to auto-repeat when held down for a predetermined lengthof time. In one embodiment, pressing the Enter key is equivalent todouble-clicking on the zone 103 corresponding to the most recentlypressed key 401.

In the example of FIG. 4B, keyboard 400 includes touchpad 405 forperforming stroke actions; these replace stroke keys 402 of FIG. 4A.Moving a finger across touchpad 405 is equivalent to a pen stroke ormouse movement, a tap is equivalent to a click, and a double-tap isequivalent to a double-click. A user can press a key 401 and stroke ortap touchpad 405 to perform a stroke action in the zone 103corresponding to the pressed key 401. Alternatively, stroke action canbe activated by pressing shift plus a key 401 and then stroking ortapping touchpad 405. For example, to capitalize first letters of eachword in the active object, a user presses Change To button 401F and thenperforms a right stroke on touchpad 405.

One skilled in the art will recognize that keys 401 and 402 may beimplemented as standard keys or buttons, or touch-sensitive regionssurrounding QWERTY keyboard 403, or by some other technique.Alternatively, a standard keyboard can be used, wherein some keys arereassigned, either temporarily or permanently, to perform commandsanalogous to zones 103. In one embodiment, keys are reassigned accordingto their physical locations on the keyboard. For example, a standard“Esc” key accesses the Start command corresponding to zone 401A; a Tabkey accesses the Format command corresponding to zone 401D; standardarrow keys can activate strokes, and the like. Alternate menus can beactivated, for example, by holding down shift key 404 while pressing thedesired zone key 401.

In another embodiment, a standard keyboard can be used, and menus 801and commands can be chosen via keystrokes that correspond to the firstletter (or some other letter) of the menu 801 or command. Arrow keys canbe used to navigate within a menu 801 once it has been displayed. Anenter key can be used to activate the currently highlighted menu item.

Using the above techniques, the present invention can be implemented ina manner that allows all input to be provided on a keyboard withoutrequiring pen, mouse, or other pointing device.

Voice Input

In one embodiment, the user interface of the present invention can becombined with existing voice-activated user interface systems. The usercan activate a command by speaking a word corresponding to the desiredcommand. Voice recognition software, as is well known in the art,interprets signals from the microphone to determine what words werespoken by the user. The command corresponding to the spoken word is thenperformed.

In one embodiment, the user activates a voice command by first speakingone of the words corresponding to a zone 103 of a control region. Thecommand menu 801 for that zone 103 is displayed, and the user canactivate a command within the menu 801 by speaking the wordcorresponding to the desired command. Alternatively, if the user wishesto bypass the menu display, he or she can simply speak the word for azone 103 followed immediately by the name of the command within the menu801 for that zone 103. For example, a user can say “Format”; the formatmenu 801 is then displayed; and the user can say “Bold” to choose theBold command from the format menu 801. Alternatively, the user cansimply say “Format Bold” to activate the Bold command.

In one embodiment, the system of the present invention also acceptsvoice input for dictation and transcription. Thus, the user can entertext by speaking into the microphone; the system interprets the spokenwords as text input. Such functionality is implemented according to wellknown voice input techniques.

Remote Control Input

In one embodiment, the system of the present invention accepts inputprovided via a remote controller. Referring now to FIG. 5, there isshown an example of a remote controller 500 according to one embodimentof the invention. Remote controller 500 transmits signals, such asinfrared or RF signals, to a device (not shown) supporting the presentinvention via transmitter 501 according to well-known techniques.

In one embodiment, remote controller 500 includes touchpad 502 that issubdivided into a number of zones 503A through 503K. Zones 503A through503H are arranged in a similar manner to zones 103A through 103K of theon-screen interface depicted in FIGS. 3A and 3B. Touching or strokingany of zones 503A through 503K effects the same result as touching orstroking corresponding zone 103A through 103K as described above.Touching a zone 503A through 503K and holding the finger for a momentprior to moving activates the zone's alternate (or secondary) command,as described above. In addition, touchpad 502 includes zone 503L whichcontrols volume (for example, by stroking up or down to increase ordecrease volume, or tapping to mute and unmute). The remote controller500 generates an appropriate signal based on the touched/stroked zone503, and transmits the signal to the device. The device has a signalreceiver (e.g., IR receiver or RF receiver), which receives the signals,converts it to the appropriate format and semantics if needed by theoperating system of the device, which provides the signal to anapplication having the user interface 100.

In another embodiment, remote controller 500 includes a number ofbuttons (not shown) that replace touchpad 502 and perform analogousfunctionality to zones 503 described above. Alternatively, one skilledin the art will recognize that other input knobs, buttons, wheels, andthe like, may be provided instead of or in addition to those describedherein. In addition, one skilled in the art will recognize that theparticular arrangement of zones 503 in FIG. 5 is merely exemplary.

Screen Shot Examples

Referring now to FIG. 6, there is shown a screen shot depicting anexample of a menu 801 for an input field 700. Menu 801 is activated byclicking on the lower right zone of control region 101. Input field 700contains the value “05:30 p-05:45 p.” Menu 801 includes commands 803 forchanging the end time by various increments. Icons 802 indicate strokedirections corresponding to commands 803. In the example, horizontalstrokes are used for coarse adjustments (by quarter-hour increments),and vertical strokes are used for fine adjustments (by one-minuteincrements).

Referring now to FIG. 8, there is shown a screen shot depicting a window800 including examples of input fields 700, 700A. Various types ofnumeric data are shown in input fields 700, 700A. The techniques of thepresent invention can be used to adjust or manipulate the values andformats for all of the examples shown, among others. One input field700A currently has focus, and the user has activated menu 801 for inputfield 700A by clicking on the lower right zone 103H of control region101. As in the example of FIG. 6, input field 700A contains the value“05:30 p-05:45 p.” Menu 801 includes commands 803 for changing the endtime by various increments. Icons 802 indicate stroke directionscorresponding to commands 803. Again, horizontal strokes are used forcoarse adjustments (by quarter-hour increments), and vertical strokesare used for fine adjustments (by one-minute increments).

Referring now to FIG. 9, there is shown a screen shot depicting anexample of a menu 801 for a table cell 902A. Table 901 includes a numberof cells 902, 902A. Cell 902A (which spans two columns of table 901)currently has focus, and the user has activated menu 801 by clicking onthe lower right zone 103H of control region 101. Menu 801 includescommands 803 for reducing or extending cell 902A span in the horizontaland vertical directions, as well as a command 803 for resetting cell902A span to one column and one row. Icons 802 indicate strokedirections corresponding to commands 803. One icon 802 is a “:” symbol,indicating that the corresponding action for direct activation of thecommand is a double-click.

Referring now to FIG. 10, there is shown a screen shot depicting anexample of a menu 801 for a table column. Table 901 includes a number ofcells 902, 902B. Cells 902B collectively form a column that currentlyhas focus. The user has activated menu 801 by clicking on the lowerright zone 103H of control region 101. Menu 801 includes commands 803for changing the width of the column and for resetting the width to anautomatic width. Icons 802 indicate stroke directions corresponding tocommands 803. One icon 802 is a “:” symbol, indicating that thecorresponding action for direct activation of the command is adouble-click. Menu 801 also includes additional commands 803A that arenot directly activatable via a stroke or action.

Referring now to FIG. 11, there is shown a screen shot depicting anexample of a menu 801 for a table row. Table 901 includes a number ofcells 902, 902C. Cells 902C collectively form a row that currently hasfocus. The user has activated menu 801 by clicking on the lower rightzone 103H of control region 101. Menu 801 includes commands 803 forchanging the height of the row and for resetting the height to anautomatic height. Icons 802 indicate stroke directions corresponding tocommands 803. One icon 802 is a “:” symbol, indicating that thecorresponding action for direct activation of the command is adouble-click. Menu 801 also includes additional commands 803A that arenot directly activatable via a stroke or action.

Referring now to FIG. 12A, there is shown an example of a control region101 for a text object 102. Text object 102 is a word within a sentence.Control region 101 includes zones 103A through 103J as described above.

Referring now to FIG. 12B, there is shown another example of a controlregion 101 for text object 102. Again, text object 102 is a word withina sentence, and control region 101 includes zones 103A through 103J asdescribed above. In the example of FIG. 12B, each zone 103 includes atext label to indicate the type of commands available from that zone103.

Referring now to FIG. 12C, there is shown another example of a controlregion 101 for text object 102. Again, text object 102 is a word withina sentence, and control region 101 includes zones 103A through 103J asdescribed above. In the example of FIG. 12C, each zone 103 includes anicon label to indicate the type of commands available from that zone103.

Referring now to FIG. 12D, there is shown another example of a controlregion 101 for text object 102. Again, text object 102 is a word withina sentence, and control region 101 includes zones 103A through 103J asdescribed above. In the example of FIG. 12D, a pop-up label 1201 (alsoknown as a ToolTip) appears when cursor 110 hovers over a particularzone 103. The pop-up label 1201 can include text or an icon to indicatethe type of commands available from the corresponding zone 103. Pop-uplabel 1201 can be configured to appear only after cursor 110 has hoveredover zone 103 for a predetermined length of time, and can further beconfigured to disappear when cursor 110 leaves zone 103 and/or after apredetermined period of time. The technique exemplified by FIG. 12Dprovides a convenient reminder of the functionality for each zone 103without unduly cluttering the display and without consuming valuablescreen real estate. Zones 103 and control region 101 can be madesmaller, as there is no need to provide adequate room for text or iconlabels within each zone 103.

In the above description, for purposes of explanation, numerous specificdetails are set forth in order to provide a thorough understanding ofthe invention. It will be apparent, however, to one skilled in the artthat the invention can be practiced without these specific details. Inother instances, structures and devices are shown in block diagram formin order to avoid obscuring the invention.

Reference in the specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least one embodimentof the invention. The appearances of the phrase “in one embodiment” invarious places in the specification are not necessarily all referring tothe same embodiment.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the discussion, it isappreciated that throughout the description, discussions utilizing termssuch as “processing” or “computing” or “calculating” or “determining” or“displaying” or the like, refer to the action and processes of acomputer system, or similar electronic computing device, thatmanipulates and transforms data represented as physical (electronic)quantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system's memories or registers or other such informationstorage, transmission or display devices.

The present invention also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but is not limited to, any type ofdisk including floppy disks, optical disks, CD-ROMs, andmagnetic-optical disks, read-only memories (ROMs), random accessmemories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any typeof media suitable for storing electronic instructions, and each coupledto a computer system bus.

The algorithms and displays presented herein are not inherently relatedto any particular computer, network of computers, or other apparatus.Various general-purpose systems may be used with programs in accordancewith the teachings herein, or it may prove convenient to construct amore specialized apparatus to perform the required method steps. Therequired structure for a variety of these systems appears from thedescription. In addition, the present invention is not described withreference to any particular programming language. It will be appreciatedthat a variety of programming languages may be used to implement theteachings of the invention as described herein.

As will be understood by those familiar with the art, the invention maybe embodied in other specific forms without departing from the spirit oressential characteristics thereof. For example, the particulararchitectures depicted above are merely exemplary of one implementationof the present invention. The functional elements and method stepsdescribed above are provided as illustrative examples of one techniquefor implementing the invention; one skilled in the art will recognizethat many other implementations are possible without departing from thepresent invention as recited in the claims. Likewise, the particularcapitalization or naming of the modules, protocols, features,attributes, or any other aspect is not mandatory or significant, and themechanisms that implement the invention or its features may havedifferent names or formats. In addition, the present invention may beimplemented as a method, process, user interface, computer programproduct, system, apparatus, or any combination thereof. Accordingly, thedisclosure of the present invention is intended to be illustrative, butnot limiting, of the scope of the invention, which is set forth in thefollowing claims.

1. In a user interface including a plurality of stroke commands for azone, a computer-implemented method for manipulating an object,comprising: responsive to a stroke in a first direction along a firstaxis initiated within a zone proximate the object, changing acharacteristic of the object by increasing a value of a parameterassociated with the object by a first predetermined increment;responsive to a stroke in a second direction along the first axisinitiated within the zone, changing the characteristic of the object bydecreasing the value of the parameter by the first predeterminedincrement; responsive to a stroke in a third direction along a secondaxis initiated within the zone, changing the characteristic of theobject by increasing the value of the parameter by a secondpredetermined increment different from the first predeterminedincrement; and responsive to a stroke in a fourth direction along thesecond axis initiated within the zone, changing the characteristic ofthe object by decreasing the value of the parameter by the secondpredetermined increment.
 2. The method of claim 1, wherein the secondpredetermined increment is of smaller magnitude than the firstpredetermined increment.
 3. The method of claim 1, wherein the secondaxis is perpendicular to the first axis.
 4. The method of claim 3,wherein one axis is vertical, and the other axis is horizontal.
 5. In auser interface including a plurality of stroke commands for a zone, acomputer-implemented method for manipulating an object, comprising:responsive to a stroke along a first axis initiated within a zoneproximate the object, changing a characteristic of the object by a firstpredetermined increment; and responsive to a stroke along a second axisinitiated within the zone, changing the characteristic of the object bya second predetermined increment different from the first predeterminedincrement; wherein the characteristic of the object is one selected fromthe group consisting of: a start position; an end position; a duration;a size; a length; a date; a time; a numeric value; a width; a height; animage cropping specification; a thickness; a decimal place location;playing speed; playing position; a leading character; a terminatingcharacter; an alignment; a rotation; a font; a style; a capitalization;a color; an opacity; a brightness; and a relative volume.
 6. The methodof claim 1, further comprising: responsive to the user input comprisinga menu activation command: displaying a menu comprising commands;accepting a second user input selecting a command from the menu; andresponsive to the menu command, changing a characteristic of the object.7. In a user interface including a plurality of stroke commands for azone, a computer-implemented computer program product for manipulatingan object, comprising: a computer-readable medium; and computer programcode, encoded on the medium, for causing a processor to perform thesteps of: responsive to a stroke in a first direction along a first axisinitiated within a zone proximate the object, changing a characteristicof the object by increasing a value of a parameter associated with theobject by a first predetermined increment; responsive to a stroke in asecond direction along the first axis initiated within the zone,changing the characteristic of the object by decreasing the value of theparameter by the first predetermined increment; responsive to a strokein a third direction along a second axis initiated within the zone,changing the characteristic of the object by increasing the value of theparameter by a second predetermined increment different from the firstpredetermined increment; and responsive to a stroke in a fourthdirection along the second axis initiated within the zone, changing thecharacteristic of the object by decreasing the value of the parameter bythe second predetermined increment.
 8. In a user interface including aplurality of stroke commands for a zone, a computer-implemented computerprogram product for manipulating an object, comprising: acomputer-readable medium; and computer program code, encoded on themedium, for: responsive to a stroke along a first axis initiated withina zone proximate the object, changing a characteristic of the object bya first predetermined increment; and responsive a stroke along a secondaxis initiated within the zone, changing the characteristic of theobject by a second predetermined increment different from the firstpredetermined increment; wherein the characteristic of the object is oneselected from the group consisting of: a start position; an endposition; a duration; a size; a length; a date; a time; a numeric value;a width; a height; an image cropping specification; a thickness; adecimal place location; playing speed; playing position; a leadingcharacter; a terminating character; an alignment; a rotation; a font; astyle; a capitalization; a color; an opacity; a brightness; and arelative volume.
 9. The computer program product of claim 7, furthercomprising: responsive to the user input comprising a menu activationcommand: displaying a menu comprising commands; accepting a second userinput selecting a command from the menu; and responsive to the menucommand, changing a characteristic of the object.